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This article presents a study of a locally adaptive vector quantization (LAVQ) 
algorithm for data compression. This algorithm provides high-speed one-pass com- 
pression and is fully adaptable to any data source and does not require a priori 
knowledge of the source statistics. Therefore , LAVQ is a universal data compression 
algorithm . The basic algorithm and several modifications to improve performance 
are discussed. These modifications are nonlinear quantization, coarse quantization 
of the codebook, and lossless compression of the output. Performance of LAVQ on 
various images using irreversible (lossy) coding is comparable to that of the Linde- 
Buzo-Gray algorithm, but LAVQ has a much higher speed ; thus this algorithm has 
potential for real-time video compression. Unlike most other image compression al- 
gorithms, LAVQ preserves fine detail in images. LAVQ’s performance as a lossless 
data compression algorithm is comparable to that of Lempel-Ziv-based algorithms, 
but LAVQ uses far less memory during the coding process. 


I. Introduction 

Data compression is the art of packing data, the process 
of transforming a body of data to a smaller representation 
from which the original or an approximation to the orig- 
inal can be computed at a later time. Most data sources 
contain redundancies such as nonuniform symbol distri- 
bution, pattern repetition, and positional redundancy. A 
data compression algorithm encodes the data to reduce 
these redundancies. 

Data compression has not been a standard feature 
in most communication/storage systems for the following 


reasons: Compression increases the software and/or hard- 
ware cost; compression/decompression is difficult to in- 
corporate into high data rate (greater than 10 Mb/sec) 
systems; most compression algorithms are not flexible 
enough to process different types of data; the unpre- 
dictability of compressed data file size presents space allo- 
cation problems. These obstacles are less significant today 
due to recent advances in algorithm development, high- 
speed very large-scale integrated circuit (VLSI) technol- 
ogy, and packet switching communications. Data com- 
pression is now a feasible option for those communication 
or storage systems for which communication bandwidth 
and/or storage capacity are at a premium. If present 
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trends continue, the volume of speech and image data in 
the near future will become prohibitively large for many 
communication Jinks or storage devices. 

A number of applications require data compression for 
efficient data storage. To facilitate fast processing, main- 
tain accurate records, and recall old records quickly, a 
number of businesses are optically scanning their docu- 
ments and saving them on magnetic media. This takes 
up large amounts of memory; efficient storage requires 
data compression. Documents are stored for archival pur- 
poses, so a short delay in retrieval (decompression) is not 
detrimental. A similar application exists in law enforce- 
ment, security, and intelligence agencies, where facial, fin- 
gerprint, and other images are kept on file for fast retrieval, 
analysis, and matching. 

Data compression is required also in limited bandwidth 
communications; the two best examples of this are video 
telephony and high-definition television (HDTV). Video 
telephony requires transmission of images over a small 
bandwidth; this can be as small as 4 kHz in the standard 
voice communication channel. Sending an image, even a 
small one, without data compression is not feasible. For 
HDTV, data compression is also needed if signals are to be 
sent digitally over a standard television channel: HDTV 
signals require roughly four times the bandwidth of stan- 
dard TV signals. High-speed algorithms which compress 
the image without substantially degrading image quality 
are requirements for both video telephony and HDTV. As 
more and more information must be transmitted over the 
same size bandwidth, data compression becomes impera- 
tive to maintain transmission rate and data fidelity. 

Vector quantization (VQ) is an efficient data compres- 
sion technique for speech and images. VQ maps a sequence 
of continuous or discrete vectors into a digital sequence 
suitable for transmission over a digital channel or storage 
in a digital medium. The goal is to reduce the volume 
of data while preserving required fidelity levels. In [1], a 
well-designed VQ scheme was shown capable of providing 
high compression ratio with good reconstructed quality. 

Unlike scalar quantization where the actual coding of 
continuous or discrete samples into discrete quantities is 
done on single samples, the input data of a VQ encoder 
are multidimensional blocks of data (input vectors). An 
important technique in VQ is the training of codebooks 
prior to transmission [1]. Extensive preprocessing is per- 
formed on sample source data to construct the codebook 
to be used in the compression session. The encoder and 
decoder must first agree on the same codebook before data 
transmission. The closeness between an input vector and 


a codeword in the codebook is measured by an appropi- 
ate distortion function. During the compression session, 
distortions between an input vector and codewords in the 
codebook are evaluated; the codeword closest to the input 
vector is chosen as the quantization vector to represent the 
input vector. The index of this chosen codeword is then 
transmitted through the channel. Compression is achieved 
since fewer bits are used to represent the codeword index 
than the quantized input data. The decoder receives the 
codeword index and reconstructs the transmitted data us- 
ing the preselected codebook. 

TYaditional VQ schemes have a few inherent disadvan- 
tages. (1) The generation of a good codebook requires a 
priori knowledge of the source data, which in practice are 
not often easily available. (2) Traditional VQ schemes are 
static schemes. They assume that the statistical proper- 
ties of the source data remain the same for all compression 
sessions and the codebooks are optimized based on this 
assumption. Real-world data tend to have varying charac- 
teristics, and static algorithms may not be efficient enough 
to process diverse sources. (3) Both codebook generation 
and codeword search for input vectors involve computing 
the distortion between input vectors and codewords in the 
codebook; these are usually computationally intensive pro- 
cesses, especially when the codebook size is large. 

A new class of data compression algorithms, locally 
adaptive vector quantization (LAVQ) algorithms, was sug- 
gested in [2] and [3]. Unlike traditional VQ algorithms, 
LAVQ algorithms do not require a priori knowledge of the 
source, nor do they require the tedious process of codebook 
generation, as the codebook is generated on the fly during 
encoding, and the decoder mimics the operations of the 
encoder to maintain an identical codebook at all times. 
This algorithm does not require a full codebook search: 
The codebook is updated after each use to maintain the 
most recently used codewords at the front of the codebook; 
a codeword which is within the error allowance is typically 
found in the top one-fifth to one-tenth of the book through 
a sequential search. This algorithm dynamically adapts to 
the local features of the source and is particularly good in 
compressing sources with varying characteristics.. 

In this article the basic algorithm suggested in [2] and 
[3] will be described. Subsequent sections will be devoted 
to the analogy of LAVQ to a vector differential pulse code 
modulation (DPCM) algorithm, improvements to the ba- 
sic LAVQ algorithm, and application to lossless data com- 
pression. The algorithm has been fully implemented in 
software; a brief description of this implementation is in- 
cluded. Experimental results on both lossy image coding 
and lossless data compression are also presented. 
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II. Basic Algorithm 

The basic LAVQ algorithm provides a simple yet effec- 
tive one-pass data compression strategy (refer to Fig. 1). 
The encoder has a codebook containing codewords (vec- 
tors) where the index of the codeword corresponds to its 
position in the codebook. A block is taken from the image 
and compared to the stored codewords; if there exists a 
codeword sufficiently close to the image block (within the 
error allowance), the index itself is sent, and that codeword 
is moved to the top of the codebook. If no such codeword 
exists, a special index is sent. This index is followed by the 
block itself. This block becomes a new codeword and is 
placed at the top of the codebook. All other codewords are 
pushed down, and if the number of codewords exceeds the 
maximum allowed, the last codeword is lost. Initially, the 
codebook may be empty or full from the previous image 
encoded. 

On the decoder side, the decoder expects an index. If 
this index is the special one denoting that a new block was 
sent, the decoder expects a block to be received immedi- 
ately following the special index; this block is placed at the 
top of the codebook and all other codewords are pushed 
down. If the codebook is already full, the last codeword 
is discarded. This new block is also placed into the image 
being built by the decoder. If the index is not one desig- 
nating a new block, then the codeword corresponding to 
the index is put into the image being built, and that code- 
word is moved to the top of the codebook. Thus, if the 
encoder and decoder start with the same codebook, they 
will have the same co debook at each step, and the image 
will be successfully sent [2-4]. 

The LAVQ strategy maintains the most recently used 
vectors in the codebook in the order of last usage; this 
allows the algorithm to efficiently code any image on the 
fly without codebook training: The algorithm needs only 
one pass of the image to code it entirely. In serial imple- 
mentation, LAVQ has time complexity 0(nm) and storage 
complexity 0(m), where n is the number of pixels in the 
image and m is the number of codewords in the codebook. 
Most of the time spent on encoding is taken by finding the 
closest codeword in the codebook and determing if that 
match is close enough. Rearranging the codebook and 
sending the required index, and possibly a new block, can 
be done quickly in serial implementation using lookup ta- 
bles, linked lists, and other software techniques. To min- 
imize the amount of time spent on codebook searching 
and to improve performance, a partial search of the co de- 
book can be used: Instead of searching serially for the best 
match, the encoder can stop searching at the first instance 
of a close-enough match, with the criterion dictated by the 
error allowance given. 


The basic LAVQ algorithm, however, has poor per- 
formance compared to traditional VQ strategies such as 
the Linde-Buzo-Gray (LBG). Several adjustments can be 
made to improve the algorithm without degrading the ad- 
vantage of one-pass high-speed implementation. Two ap- 
proaches can be used to improve rate. First, the statistics 
of the coded indices can be skewed toward small values (re- 
cent indices) by (1) a partial codebook search as outlined 
above, (2) using tall and narrow blocks (N x 1 pixels) to 
make each block more similar to the blocks immediately 
previous to it in a raster scan, and (3) coding only the dif- 
ferential value of each block by removing the mean value 
and reinserting it later. Second, the number of bits used 
to send new codewords can be reduced by (1) reducing 
the number of bits used to describe each new pixel and (2) 
using nonlinear quantization of these new codeword val- 
ues. These two approaches, combined with lossless adap- 
tive arithmetic coding of the output, improve the perfor- 
mance of LAVQ to be comparable to that of LBG. These 
improvements will be discussed in detail in a subsequent 
section. 


III. LAVQ as a Vector Analogy of DPCM 

Differential pulse code modulation (DPCM) data com- 
pression algorithms are efficient and have low complexity. 
They are particularly effective in encoding gray-scale im- 
ages, which are dominantly characterized by an autore- 
gressive (AR) stochastic model or an autoregressive mov- 
ing average (ARMA) model. DPCM operates on individ- 
ual samples x(n) and encodes the quantized difference e(n) 
between a predicted value x(n) and x(n). The prediction 
is based on the pixels neighboring x(n). The error e(n) 
tends to be small rather than large, and compression is 
achieved by assigning fewer bits to smaller values of e(n) 
and more bits to larger values of e(n). 

From an information theoretic point of view, given a 
data source, it is always advantageous to encode vector 
quantities rather than scalar quantities. A vector exten- 
sion of DPCM coding involves encoding a vector of dif- 
ferences E = [ei(rc), e 2 (n), ..., ejv(tt)]i where N is the 
vector size. However, the number of combinations of E in- 
creases exponentially with N\ especially for large V, this 
quickly becomes too large to be practically feasible for effi- 
cient encoding. Another drawback of DPCM is its inflexi- 
bility: DPCM performs well when coding sources are char- 
acterized by the AR or ARMA models (e.g., speech and 
image data); however, for data sources dominantly charac- 
terized by pattern repetitions (e.g., data base records and 
engineering data), DPCM performs poorly. 
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LAVQ is analogous to DPCM in a sense: Both DPCM 
and LAVQ can be viewed as consisting of a preprocess- 
ing stage and a compression stage. DPCM preprocesses a 
sample by taking the difference between the sample value 
and its corresponding predicted values and sends the quan- 
tized difference to be entropy coded. LAVQ, on the other 
hand, preprocesses a vector of samples by matching it to 
the codeword vectors in a dynamic codebook followed by a 
move-to- front codebook update, and sends either a code- 
book index or an uncoded vector to the decoder. The ma- 
jor difference is that DPCM encodes the scalar difference, 
whereas LAVQ encodes the vector recency, which can be 
considered as a different measure of difference (vector dif- 
ference). Thus, LAVQ uses the locally adaptive move-to- 
front preprocessing unit to convert the hid den stat istical 
and correlational redundancy to a scalar statistical redun- 
dancy. This allows representation of vectors as scalars and 
approaches vector entropy, which, in the information the- 
oretic sense, is smaller than the corresponding scalar en- 
tropy. This was proven in [2] and is verified in the results 
given here. 

IV. Improvements to LAVQ for Image 
Compression 

A. Index Coding 

1. Difference Coding. The most significant visual 
artifact of LAVQ is the sawtooth or staircase effect. This 
occurs from using discrete vectors to represent a set of vec- 
tors within a threshold set by the error allowance; thus, 
differences between codewords can be large enough to be 
noticeable. In particular, if the pixels are slowly varying 
across the image, as is the case with most images, the 
encoded blocks do not track this variation closely. That 
is, adjacent blocks are coded with the same block while 
the amount of error is within the allowance; when that 
allowance is exceeded, suddenly a different block is used. 
This difference can be noticeable and, since most images 
have regions of slow variations or of constant color or in- 
tensity, quite common. 

This assumption of images having regions of slowly 
varying or constant pixels implies that adjacent blocks 
have similar mean values. Thus, the mean can be removed, 
and only the differential values of the image can be coded. 
Eaclvvector now represents the difference between the ac- 
tual pixel value and a reference value equal to the distorted 
mean of the previously coded block. This distorted mean 
is a valid choice of reference because both the encoder and 
decoder can compute it exactly from the previous block’s 
distorted pixel values. The small difference vectors are 


more likely to be well approximated by the recently occur- 
ring difference vectors maintained in the current codebook. 
Thus, slowly varying or constant regions can be more ac- 
curately coded without extensive use of new codewords. 
The mean must be updated after each block is processed 
at both the encoder and the decoder. This ensures that 
both have the same mean to remove and to reinsert into 
the image at each step. 

2. One-Pass Index Compression. Because of the 
move-to-front codebook rearrangement strategy and be- 
cause most images have similar adjacent blocks, the small- 
est indices are most likely to be used more often. There- 
fore, they can be coded using a lossless compression code 
to obtain better performance. However, to maintain one- 
pass compression, the lossless code must also be one-pass. 
Furthermore, the statistics of the coded indices are un- 
known a priori, and no assumption can be made regarding 
them. 

The code which yields the most promising results with 
minimal increase in computational complexity is the adap- 
tive arithmetic code. This algorithm is the static arith- 
metic code implemented with probabilities of each symbol 
updated after each use. By starting with the same initial 
distribution at the encoder and decoder, lossless coding 
can be obtained. The arithmetic code approaches global 
symbol entropy closely; in some cases, it does even bet- 
ter: The average entropy of the adaptive arithmetic code 
is average local entropy based on symbol statistics from 
the start to the symbol being coded. Global entropy is 
derived from the statistics of each symbol based on the 
entire sequence; local entropy is derived from the statistics 
of each symbol based on part of the sequence in the neigh- 
borhood of the symbol being coded. The global entropy 
is always greater than or equal to the global average of all 
the local entropies. Thus, for sources which have localized 
characteristics which vary throughout the sequence, using 
localized adaptive coding methods is more advantageous 
than using global, nonadaptive coding methods. 

B. Codeword Data Coding 

1. Bit Stripping. Bit stripping of new codeword val- 
ues can be used independently from or in conjunction with 
difference coding to obtain higher compression rates. The 
least-significant bits of either a block of pixels or a vec- 
tor of differences tend to be uniformly random; therefore, 
stripping these bits before sending data to the decoder 
and reinserting a mean value at the decoder decreases the 
number of bits sent when a new codeword is generated 
for the codebook, with a small increase in distortion. The 
amount of additional error incurred is not readily notice- 
able in most images. 
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2. Nonlinear Quantization. When difference coding 
is used with bit stripping, only a few values typically occur, 
and these may be represented by a relatively small number 
of quantization levels. However, there may be sharp edges 
in the image which will have large differences in value be- 
tween adjacent blocks. This can cause large errors at edges 
if linearly quantized difference values are too small to keep 
up with the rapid change in pixel values. To minimize this 
error, nonlinear quantization can be used. The choice of 
quantization step sizes is not clearly defined, as the im- 
age statistics are unknown to the encoder and cannot be 
easily transmitted to the decoder. Therefore, design of 
the quantizer cannot be made adaptive, and an arbitrary 
choice must be made beforehand. Quantizer design has 
some criteria, however. Initial step sizes (near zero) should 
be small, and subsequent sizes should increase. Based on 
this assumption, a fixed logarithmic quantizer is used. 

C. Interpolation and Smoothing 

As mentioned earlier, since LAVQ uses discrete vec- 
tors to represent a set of vectors within a threshold set by 
the error allowance, there is noticeable blockiness in the 
output. Difference coding helps remove this effect, but it 
is insufficient: Since zeroth order estimation of the next 
block is used — the mean is assumed to be identical across 
adjacent blocks— regions with gradual pixel value changes 
can cause a sawtooth or staircase visual artifact at high 
compression rates. In addition, difference coding does not 
remove the block boundaries visible in the vertical direc- 
tion. In cases where difference coding is used and where it 
is not, horizontal interpolation can remove the horizontal 
artifacts, and vertical smoothing can remove the vertical 
artifacts; if done carefully, both techniques do not exces- 
sively destroy detail or cause a blurry appearance. 

Horizontal interpolation essentially interpolates across 
those blocks represented by different codewords. Each 
block is classified as either a repeat of the previous block 
or not; the first occurrence of a block that is not a repeat 
of the previous block is recorded; the rest are initially left 
blank. These blank blocks are filled with a pixelwise lin- 
ear interpolation of the edges of the two closest nonblank 
blocks. However, this can smear edges which occur in the 
image; therefore, a threshold is used: If the difference be- 
tween two differing blocks is larger than this threshold, 
no interpolation is done. This threshold must be adjusted 
externally. 

Vertical smoothing averages the pixels on the vertical 
block borders. If both blocks on the border are not new 
codewords, that is, both already exist in the codebook, 
then the two border pixels are averaged; this average value 


is substituted for the border pixels. If both blocks are new 
codewords, nothing is done. If only one is new and the 
other is an existing codeword, then the new block is un- 
altered; the existing block’s boundary pixel is substituted 
with the average of the boundary pixel of the new block 
and the two boundary pixels (the boundary and the pixel 
vertically adjacent to it) of the existing block. In this way, 
new codewords, which usually describe detailed areas, are 
unaltered, while existing codewords, which describe areas 
of low detail, are smoothed. 

V. Lossless Data Compression 

As mentioned earlier, the LAVQ algorithm is also suit- 
able for lossless compression of database records and other 
data dominantly characterized by pattern repetitions. Ex- 
amples of these data sources include textual data, account- 
ing and payroll database data, telemetry data, and engi- 
neering data. Lossless compression using LAVQ can be 
achieved easily by setting the error allowance to zero. In 
the lossless compression mode, the basic LAVQ encoder 
becomes a locally adaptive move-to-front (MTF) algo- 
rithm. 

Lossless LAVQ works best on data with fixed record 
sizes. Data represented in fixed-size packets and with 
patterns confined to the packets or fixed subfields within 
them are the best candidates for LAVQ. However, ar- 
bitrary fixed-length blocking of data can also be used 
on other sources without defined block sizes without sig- 
nificant detriment. Universal data compressors such as 
Lempel-Ziv (LZ) -based algorithms assume no structure 
of the source except for intersymbol correlation; however, 
many of these algorithms require large amounts of mem- 
ory during coding, use complex tree data structures such 
as the “Patricia tree” to maintain their dictionaries, and 
have sophisticated pruning techniques to update the code 
tree. LAVQ requires relatively less memory and less com- 
plex data structures; it uses a simple codebook updating 
algorithm (MTF in this case) with a much smaller data 
buffer. Several high-performance LZ-based algorithms use 
back-end entropy coders to further improve performance; 
LAVQ can be likewise equipped. In short, lossless LAVQ 
can achieve rates comparable to the best LZ-based algo- 
rithms. 

VI. Software Implementation 

Software implementation of LAVQ, complete with all 
improvements, has been completed. To minimize source 
code complexity, the arithmetic coder has been imple- 
mented separately from the encoder. Parameters variable 
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in the encoder include block dimensions, codebook size, 
error tolerance, number of bits stripped, first or best oc- 
currence of an acceptable codeword, and difference cod- 
ing. Furthermore, to allow use of the encoder for image 
sequences, as in video, codebook preservation is also pro- 
vided. This allows the codebook from the previous frame 
to be used in the subsequent one. 

A. LAVQ Encoder and Decoder 

The encoder program first reads in all the data, then 
converts the image into a linked list of blocks. The code- 
book is specified as a doubly linked list to facilitate speed 
in rearrangement: Only ten pointers at most need be 

changed to do a complete codebook rearrangement. The 
codebook is initially assumed empty. At each step, a block 
is compared with the codebook entries and the best or first 
occurrence of an acceptable codeword is found. The code- 
book is updated as needed, and the requisite values are 
output. The program can also calculate global entropy of 
the encoded file to give an estimate of the highest compres- 
sion possible with these parameters. The decoder program 
reverses this arrangement, with each input being an index 
or a new codebook value, and the codebook is rearranged 
in a manner identical to the encoder codebook. The image 
is rebuilt as a linked list and is then converted to image 
format and output. 

B. Adaptive Arithmetic Coding 

The arithmetic coder implemented is one described in 
[5]; it is used here with only minor modifications; the most 
notable of these is the ability to input symbols of alphabet 
size less than 256. The assumption is made at first that the 
symbols are all equiprobable; after each symbol is encoded, 
its statistics are modified to reflect this. Two encoders and 
decoders are used: One pair encodes and decodes only the 
indices from the output of the LAVQ encoder and ignores 
the new codeword information; the second pair codes and 
decodes these new codeword values. 

The basic arithmetic code operates in the following 
manner: The symbols are arranged in some order and 
are assigned regions of size corresponding to their prob- 
abilities. These regions span the space from 0 to 1. The 
coding region is initially defined to be [0,1). When a sym- 
bol is coded, the symbol space [0,1) is scaled to fit the 
coding region, and the new coding region is defined by the 
region specified by the symbol coded. Therefore, at each 
step, the coding region becomes more and more narrow. In 
practice, the coding regionjs scaled in size as each tiffafn- 
biguous most significant bit is transmitted. The decoder 
reverses this by scaling up the coding region as symbols 


are decoded. A detailed discussion of arithmetic codes is 
available in [5]. 

VII. Experimental Results 

A. Image Compression 

The LAVQ algorithm was tested on a number of mono- 
chrome 8-bit 512 x 512 pixel images. Global pixel en- 
tropies, which are entropies estimated over the whole im- 
ages, are listed in Table 1. These images were selected to 
provide a diverse cross section of images to examine the 
flexibility of LAVQ. A portrait (“lena”), a wildlife/natural 
scene of a seal on a rocky seashore (“seal”), a high de- 
tail overhead view of Los Angeles International Airport 
(“lax”), the cratered surface of Mercury (“mercury”), the 
rings of Saturn (“saturn”), and a medical CAT scan image 
(“catOl”) were used. The images are shown in Fig. 2. 

Performance parameters are measured in mean squared 
error (MSE) for distortion and required bits per pixel for 
rate. MSE is defined as 

1 5123 

MSE 5Y22 ^ v [^original,! ^processed,*'] 
i= 1 

for images of size 512 x 512 pixels. 


The LAVQ parameters used were 8x1 blocks with 
255 codewords in the codebook. Difference coding was 
used, and 4 bits with logarithmic quantization levels were 
used to represent each new codeword value. There is not 
much difference in performance between finding the best 
match in the codebook (complete codeword search) and 
stopping after the first instance of an acceptable (within 
error allowance) codeword in the book (partial codebook 
search, typically one-fifth to one-tenth of the entire book); 
therefore, the latter strategy is used to maximize speed 
and to favorably skew the index statistics. Codebook and 
block sizes were selected to obtain good results; block sizes 
larger or smaller than 8 x 1 yielded worse results, and in- 
creasing codebook size beyond 255 had only marginal rate 
improvement with substantial increase in computational 
complexity. Block interpolation and smoothing are used 
with a threshold of 32. The LBG algorithm using blocks 
of size 4x4 pixels (found to yield good results in general 
and better results than LBG with 8 x 1, 4 x 2, or 16 x 1 
pixel blocks) is presented for comparison. ~ T 

In the rate-distortion curves generated for all six images 
(see Figs. 3 through 8), the LAVQ blocks are 8 x 1 pixels; 
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255 codewords are in the codebook, the difference coding 
has 4 bits per codeword value, and there are logarithmic 
quantization, block interpolation, and smoothing, except 
for Fig. 5, which lacks block interpolation and smoothing. 
LBG was done with 4x4 pixel blocks. LAVQ sends the 
codebook simultaneously with the indices while compress- 
ing the image; therefore, a true comparison should include 
the cost of sending the codebook for LBG as well.. The 
LBG curves were generated using 4x4 pixel blocks and 
varying codebook sizes (from 16 to 8192). LAVQ curves 
were generated using the parameters outlined above. The 
curves have differing scales because each image has differ- 
ent characteristics which alter the algorithms’ ability to 
compress them. 

In most cases, LAVQ does better (defined as having a 
lower distortion for a given rate or vice versa) than LBG 
in the low-distortion regions (high-rate regions); in this 
region, LAVQ sends more new codewords, and these new 
codewords are more accurate renditions of the original im- 
age than the codewords used by LBG, which are the cen- 
troids of many blocks. Because LAVQ requires that more 
codewords be sent than LBG for low-distortion cases, this 
factor by itself would seem to make the rate for LAVQ 
worse than for LBG. However, LAVQ can take good ad- 
vantage of lossless entropy coding of indices and new code- 
words. In contrast, because LBG’s codebook search algo- 
rithm distributes its codewords to span the space in which 
the image’s vectors exist, in general LBG does not profit 
as much as LAVQ does from lossless compression. Only 
about a 5-percent improvement using arithmetic coding 
was noted, compared to over 50-percent improvement ob- 
tained for LAVQ using arithmetic coding of both indices 
and codewords. As a result, LAVQ often has the potential 
for better performance than LBG after entropy coding is 
applied to LAVQ. 

At lower rates (and higher distortions), LBG takes time 
to select a good codebook and therefore can do much bet- 
ter than LAVQ, which does not train a codebook at all. 
This is the dominant factor which can make LAVQ per- 
formance worse than LBG: Since LAVQ does not train 
a codebook, it relies on recently occurring vectors for a 
source of new codewords. These do not necessarily reflect 
a good choice of codewords, so for a given rate, the distor- 
tion for LAVQ can be higher than for LBG. 

Such generalizations, however, have many exceptions. 
For example, two images in the set used here, “catOl” and 
“saturn,” have large regions of uniformity compared to 
the other images. In the high-rate (low-distortion) region, 
these low-detail regions are better coded by LBG than 
by LAVQ because the LBG codewords can code regions of 


low detail with less distortion; this effect is enough in these 
two images to defeat the advantage that LAVQ gains by 
lossless compression of the indices and codewords. 

In the low-rate (high-distortion) region, “catOl” and 
“saturn” are better coded with LAVQ than with LBG. 
With other images, the time spent by LBG in developing 
a good codebook made the performance of LBG better 
than what LAVQ could achieve without training. With 
these two images, however, LAVQ can code the low-detail 
regions quite easily with very few new codewords while the 
details are better preserved in the high-detail regions. This 
can be enough to offset the disadvantages mentioned ear- 
lier that LAVQ encounters in the low-rate (high-distortion) 
region. 

Each LBG rate-distortion curve of 10 data points took 
approximately 100 hours of computation on a Sun Sparc 2, 
while one LAVQ curve with 24 data points, complete with 
lossless compression, was done in about 1 hour: LAVQ is 
much faster than LBG and can achieve performance com- 
parable to LBG if the cost of the codebook is included. 
Furthermore, LAVQ preserves detail better than LBG, 
but does so with increased blockiness in low-detail regions. 
This detail-preserving feature of LAVQ is discussed in the 
next section. 

B. Detail Preservation 

As mentioned in the previous sections, LAVQ operates 
by matching a vector to the codewords in the codebook us- 
ing a predetermined fidelity criterion, and new codewords 
are entered verbatim from the examined vector if no match 
occurs within this error allowance. Therefore, those vec- 
tors which are significantly different from previous vectors 
are coded without distortion; these occur at edges or areas 
of high detail. The cost of preserving these details, how- 
ever, is increased distortion in low-detail (“smooth”) ar- 
eas. Codewords are not optimized to best represent these 
regions, so they exhibit more blockiness. Thus, LAVQ pre- 
serves details and is potentially very attractive to military 
intelligence and space applications: These applications re- 
quire close examination of details to identify and differen- 
tiate among various objects. 

These effects are illustrated in Figs. 9 and 10. In Fig. 9, 
the upper right edge of the hat brim of “lena” is shown. 
The LBG codebook size, to be comparable to LAVQ, is 
fixed at 256. The LAVQ error allowance was adjusted to 
yield a distortion comparable to LBG. LBG achieves 43.83 
MSE at 0.50 bits/pixel (compression ratio of 16:1). LAVQ 
achieves 43.68 MSE at 0.56 bits/pixel (compression ratio 
of 14:1). The edge of the hat brim is rendered with much 
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less blockiness with LAVQ; the drawback is that the low- 
detail areas exhibit noticeable blockiness. 

In Fig. 10, the lower right edge of the upper terminal 
of “lax” is shown. The LBG codebook size is again fixed 
at 256. LBG achieves 166.8 MSE at 0.50 bits/pixel (com- 
pression ratio of 16:1). LAVQ achieves 166.0 MSE at 0.77 
bits/pixel (compression ratio of 10:1). Here, LBG does 
poorly in preserving the details of the aircraft, terminal, 
and service vehicles; details of the terminal and several ser- 
vice vehicles have disappeared. LAVQ does much better 
on these details, but exhibits more blockiness than LBG 
when representing the tarmac, which has less detail. With 
LAVQ, it is possible to identify aircraft type, while it is 
more difficult with LBG. For the two aircraft in Fig. 10, 
the fuselage and wing shapes, engine locations, and other 
details are preserved more clearly by LAVQ than by LBG. 

C. Lossless Data Compression 

Demonstration of lossless compression of structured 
data was conducted on Magellan space-probe engineer- 
ing data. The Magellan spacecraft engineering data file 
“mdata” consists of 1692 records of 100 bytes each; each 
record consists of 15 fields of various sizes ranging from 
4 bytes to 16 bytes. Many good existing LZ variants can 
achieve compression ratios for “mdata” of 2.2:1 to 5:1 of 
the original file size, depending on the amount of mem- 
ory consumed. The UNIX program “compress” achieves a 
compression of 3:1 at a cost of 544 kbytes of memory used. 

The Magellan engineering data have both natural pars- 
ing (a frame marker at the beginning of each record) and 
artificial parsing (the record size is fixed, 100 bytes) prop- 
erties. Most real-world data sources like texts, images, and 
engineering data records possess either or both of these two 
parsing properties. To apply LAVQ to these data, the file 
was first blocked in sequential order into 5- byte blocks. 
LAVQ was then applied to each block position; thus, all 
of the first 5-byte blocks (bytes 1 to 5 in the 100-byte 
record) were coded using one encoder, all of the second 
5-byte blocks (bytes 6 to 10 in the 100-byte record) were 
coded using a second encoder, and so on. 

The resulting LAVQ output, both indices and new 
codeword values, was then coded with the adaptive arith- 
metic coder used earlier. Results using a Q-coder [6,7] and 
the theoretical limit reached by nonadaptive means (com- 
puted from the global entropy) are also included for com- 
parison. The Q-coder is essentially an adaptive arithmetic 
coder; it differs from the arithmetic coder used above: The 
arithmetic coder maintains a running total of frequency of 
use of each symbol and uses this to determine probabilities 


for arithmetic coding. The Q-coder uses a fixed probability 
table accessed by an adaptive finite state machine. This 
finite state machine adapts with the previous symbols; its 
state is determined by the bits in each byte. 

Two differing approaches are taken. First, the code- 
book size is fixed at 256, and only one coder is available. 
Thus, a large buffer is needed to store the data to allow 
sequential coding of each of the 20 blocks in each record. 
Results tabulated in Table 2 show that the best perfor- 
mance is obtained from using large buffers: Larger files 
have more opportunities for repetitions and patterns of 
codeword usage. In Table 2, compression ratios of Magel- 
lan engineering data are of 169,200-byte size. Tabulated 
are compression ratios achieved by the adaptive arithmetic 
coder used for LAVQ, the Q-coder, and the theoretical 
maximum using a global entropy coder. This last value 
is derived from the global per-symbol entropy. Note that 
larger buffer sizes provide greater compression. Arithmetic 
coder compression approaches the theoretical value (global 
entropy) closely. Even better results are sometimes ob- 
tained. The Q-coder and arithmetic coder sometimes ap- 
pear to have results better than entropy; this occurs be- 
cause these adaptive coders code on the basis of local en- 
tropy (entropy over a localized window of characters) as 
opposed to the global entropy listed here. The Q-coder 
does better than arithmetic coding, which does about as 
well as theoretically possible with a global entropy coder. 

In the second approach, the codebook size is allowed 
to vary, and all 20 coders are available for parallel cod- 
ing. No additional buffer is needed in this case, as the 20 
blocks are coded simultaneously. Again, the indices and 
new codeword values are coded with the adaptive arith- 
metic coder and compared with the Q-coder and the global 
entropies; results are listed in Table 3. In this table, as in 
Table 2, compression ratios of Magellan engineering data 
are of 169,200-byte size. No dataT>uffer is allowed here, 
but 20 coders are operating in parallel. In this case, the 
arithmetic coder does slightly better than the Q-coder. 
Again, the Q-coder and arithmetic coder sometimes ap- 
pear to have results better than entropy for the same rea- 
son given for Table 2. Best performance is obtained with 
larger codebooks; large codebooks can record codewords 
farther into the past and therefore have more opportuni- 
ties for codeword matching. Here, the arithmetic coder 
does better than entropy and the Q-coder. In both cases, 
LAVQ performance is comparable to LZ-based algorithms. 
The advantage of LAVQ is that far less memory is required 
than for LZ algorithms; this is of importance in systems 
with size, weight, or complexity constraints, as is the case 
with deep-space probes. 
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VIII. Conclusion 

The LAVQ algorithm provides a fast, one-pass data 
compression algorithm. Improvements to the basic algo- 
rithm maintain this one-pass high-speed property while 
increasing performance measurably. Experimental results 


in image compression yield performance not significantly 
inferior to LBG, but at a fraction of the complexity. Dis- 
tortion in images occurs as blockiness in low-detail areas, 
while high-activity areas maintain sharp details. LAVQ 
also performs well in lossless compression, again with low 
complexity as compared with other algorithms. 
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Table 1. Pixel entropies. Global pixel entropies 
of Images. 


Images 

Entropy, bits/pixel 

“catOl” 

5.503 

"lax” 

6.827 

‘lena” 

7.445 

"mercury” 

6.416 

"satum” 

6.885 

"seal” 

7.356 


Table 2. Magellan data compression: sequential compression. 


Buffer 
size, bytes 

Codebook 

size 

Compression ratio achieved by 

Arithmetic 

coder 

Q- coder 

Global 

entropy 

169200 

256 

3.957 

5.053 

3.887 

84600 

256 

3.752 

4.764 

3.748 

56400 

256 

3.635 

4.556 

3.640 

42300 

256 

3.429 

4.210 

3.440 

28200 

256 

3.285 

3.929 

3.296 

14100 

256 

2.913 

3.320 

2.921 


Table 3. Magellan data compression: simultaneous compression. 


Co debook 
size 

Compression ratio achieved by 

Arithmetic 

coder 

Q-coder 

Global 

entropy 

256 

3.957 

3.912 

3.887 

128 

3.873 

3.815 

3.798 

64 

3.411 

3.290 

3.314 

32 

2.973 

2.814 

2.949 

16 

2.747 

2.545 

2.6076 
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: *- ENCODER 


DECODER 


CODEWORD EXISTS IN CODEBOOK: 



CODEWORD DOES NOT EXIST IN CODEBOOK: 


NEW NEW 

CODEBOOK CODEBOOK CODEBOOK CODEBOOK 



Fla 1 Encoder and decoder: An Image block Is compared to the codebook (A); it a codeword close enough 
exists, that codeword Is moved to the top of the codebook (B) and the Index Is transmitted (C). If It does not exist 
(D) then the block Is Inserted at the top of the codebook (E) and the Index m + 1 and the block are transmitted 
(R. On the receiver side, If an Index Is received, the corresponding codeword (G) Is moved to the top of the 
codebook and the block Is inserted into the reconstructed Image (H). If the special Index m + 1 is received (I), a 
raw block is anticipated Immediately following; this block is placed In the codebook and also in the reconstructed 

image (J). 
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"saturn" "seal" 

F(g. 2. Original images. All images are 512 X 512 pixel, 8-bit monochrome. 
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Fig. 5. Rate-distortion curve for *iena." 














ORIGINAL 


LBG 


LAVQ 


Fig. 9. Detail of "lena." Note that LBG has more blocklness at the edge, but represents low-detail {“smooth”) areas without as much 

blocklness as LAVQ. 





ORIGINAL 


LBG 

Fig. 10. Detail of "lax.” 
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